{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": [
     "remove-cell"
    ]
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "import os\n",
    "if not any(path.endswith('textbook') for path in sys.path):\n",
    "    sys.path.append(os.path.abspath('../../..'))\n",
    "from textbook_utils import *"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(ch:text)=\n",
    "# Working with Text"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data can reside not just as numbers but also in words: names of dog breeds,\n",
    "restaurant violation descriptions, street addresses, speeches, blog posts, internet reviews, and much more.\n",
    "To organize and analyze information contained in text, we often need to do some of the following tasks:\n",
    "\n",
    "Convert text into a standard format\n",
    ": This is also referred to as\n",
    "  *canonicalizing text*. For example, we might need to convert characters to\n",
    "  lowercase, use common spellings and abbreviations, or remove punctuation and\n",
    "  blank spaces.\n",
    "\n",
    "Extract a piece of text to create a feature\n",
    ": As an example, a string might\n",
    "  contain date embedded in it, and we want to pull it out from the string\n",
    "  to create a date feature.\n",
    "\n",
    "Transform text into features\n",
    ": We might want to encode particular words or phrases as 0-1 features to indicate their presence in a string.\n",
    "\n",
    "Analyze text\n",
    ": In order to compare entire documents at once, we can\n",
    "  transform a document into a vector of word counts.\n",
    "\n",
    "This chapter introduces common techniques for working with text data. We\n",
    "show how simple string manipulation tools are often all we need to put text in\n",
    "a standard form or extract portions of strings. We also introduce regular\n",
    "expressions for more general and robust pattern matching. To demonstrate these\n",
    "text operations we use several examples. We first introduce these examples and describe\n",
    "the work we want to do to prepare the text for analysis. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
